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Abstract —In this paper, we develop an extremum seeking 
control method integrated with iterative learning control to track 
a time-varying optimizer within finite time. The behavior of 
the extremum seeking system is analyzed via an approximating 
system - the modified Lie bracket system. The modified Lie 
bracket system is essentially an online integral-type iterative 
learning control law. The paper contributes to two fields, namely, 
iterative learning control and extremum seeking. First, an online 
integral type Iterative learning control with a forgetting factor is 
proposed. Its convergence is analyzed via fc-dependent (iteration- 
dependent) contraction mapping in a Banach space equipped 
with A-norm. Second, the Iterative learning extremum seeking 
system can be regarded as an iterative learning control with 
“control input disturbance”. The tracking error of its modified 
Lie bracket system can be shown uniformly bounded in terms of 
iterations by selecting a sufficiently large frequency. Furthermore, 
it is shown that the tracking error will finally converge to a 
set, which is a A-norm ball. Its center is the same with the 
limit solution of its corresponding “disturbance-free” system (the 
iterative learning control law); and its radius can be controlled 
by the frequency. 

Index Terms —Contraction mapping, extremum seeking (ES), 
iterative learning control (ILC), Lie bracket, A-norm 


I. Introduction 

C ONSIDERABLE research efforts have been devoted to 
extremum seeking control for the last several decades. 
The mechanism of extremum seeking control is to optimize 
a certain system performance measure (cost function) by 
adaptively adjusting the system parameters merely according 
to output measurements of the plant. Since there is little knowl¬ 
edge required about the plant dynamics, extremum seeking 
control has attracted the attention from various engineering 
domains, e.g., bioreactor, combustion, compressor ||1|-Q. In 
most of the classic extremum seeking literatures, the basic 
assumption is that the static input-output mapping is time 
invariant, i.e., @,0,0 and the references therein. However, 
time varying input-output mapping is common in practice. 
For example, injection molding engineers are quite interested 
in finding an optimal ram velocity profile to minimize the 
unevenness of polymer melt front velocity (PMFV) in the mold 
cavity to yield a smooth surface on the final product. The map 
from ram velocity to PMFV is typically time varying, due to 
the complex geometry in the cavity 0. 
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In this paper, we propose a novel extremum seeking scheme 
based on iterative learning control to find the optimizer (op¬ 
timal trajectory) of a time varying mapping. Particularly, the 
contributions of the paper are threefold and summarized as 
follows. 

First, a novel extremum seeking approach is developed to 
solve the optimizer tracking problem by introducing an analog 
memory into the extremum seeking loop, which differs from 
the existing methods 0.0-ini- There are two categories 
of similar approaches concerning the time varying mappings 
reported in literatures. Wang and Krstic introduced a detector 
to minimize the amplitude of stable limit cycle by tuning a 
controller parameter to a constant optimizer 0 . Guay and his 
colleagues employed system flatness to parameterize all the 
variables by sine and cosine series; extremum seeking was 
used to steer the coefficients of the series to the optimizer 
0. Haring and his coworkers have developed a mean-over- 
perturbation-period filter to produce an estimate of the gradient 
for extremum seeking loop 0 - The underlying assumption 
of all these methods is that the corresponding cost functions, 
although time varying, admit a constant optimizer, which 
is different from our discussion in this paper. As for the 
second type, Krstic introduced a compensator for time varying 
mapping which is structured by Wiener-Hammerstein models. 
This method requires the knowledge about the two time 
varying blocks, which may restrict its applicability 0, ng. 
An adaptive delay-based estimator was introduced to feed gra¬ 
dient estimates to extremum seeking loop by Sahneh and his 
coworkers. The extremum seeking loop is a cascade feedback 
in nature 0- Basically, this type of methods employed a 
fast decay ratio to suppress the tracking error. However, in 
some circumstances, these methods may not be able to yield 
satisfying transient tracking performance. Fortunately, many 
time varying systems exhibit certain repetitive behaviors, such 
as semiconductor manufacturing, pharmaceutical producing 
and the injection molding mentioned above GD- Exploiting 
such repetitiveness provides potential for circumventing the 
aforementioned drawbacks. Actually, the works in 0 
have already utilized such a feature but not that sufficiently, 
since they only used it to reduce the cost function into a 
starting-time-independent function. Iterative learning control, 
first proposed by Arimoto et. al. is good at exploiting 
repetitiveness to improve tracking performance from iteration 
to iteration GD, G3-G3- 

Second, the proposed approach is a new ILC scheme. The 
fundamental assumption adopted in most ILC literature is that 
the tracking trajectory has been already available as priori 
knowledge which renders the controller knowing the 
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direction to steer. Within this paper, we do not have such 
an assumption; only the distance to the tracking trajectory is 
known, i.e., the absolute value of tracking error. Intuitively, 
almost all the ILC approaches will fail for this situation, since 
somehow the negative feedback along the iterations cannot be 
ensured. It is natural to come up with combining extremum 
seeking with ILC; let them collaborate with each other; ILC 
provides the past learning experience to extremum seeking 
to improve transient tracking performance; the “direction” 
information needed by ILC is given by extremum seeking. 


Third, we propose a new online infinite-dimensional 
integral-type (I-type) ILC control law. The dynamical behavior 
of the ILC system is studied by A:-dependent contraction map¬ 
ping. The convergence of the ILC system is discussed. Similar 
results are |^-@.|[T 8 ) has studied the proportional-type (P- 
type) online ILC, but only derived an index bound regarding 
the ultimate tracking performance. Wang has considered the 
sampling effect and input saturation issues in the offline P-type 
ILC, and implemented it experimentally | fT9| . In | |20) , Saab 
has investigated the offline P-type and D-type (derivative-type) 
ILC for the stochastic scenario, where a dynamic learning 
gain was adopted. | 2 I| has discussed the forgetting factor 
selection for a general offline ILC algorithm. Ouyang and 
his colleagues have developed an online PD-type ILC for a 
class of input-affine nonlinear system, and also presented an 
ultimate bound of tracking performance p2) . According to 
authors’ knowledge, there is few papers contributing to online 
I-type ILC. Furthermore, we present more than an ultimate 
bound of tracking performance; the limit solution and its 
uniqueness are studied as well. More interestingly, the nature 
of the proposed extremum seeking system has been revealed, 
when the behaviors of the proposed methods are analyzed via 
an approximating system - modified Lie bracket system. It is 
an online integral-type ILC with “control input disturbance”. 
Based on the same reasoning, we extend the results on ILC to 
analyze the iterative learning extremum seeking system. We 
show that the system is uniformly bounded in terms of k and 
converges to a set as k goes to infinity. The particular set is a A- 
norm ball, whose center is the limit solution of the associated 
ILC system, and radius can be controlled by the frequency of 
the sinusoid signal. 


The rest of the paper is structured as follows: Section 
II gives the technical preliminaries about A-norm and Lie 
bracket system; Section III formulates the problem; Section 
IV presents the main results; Section V provides an illustrative 
example for the theory; a conclusion is drawn and an outlook 
is given in Section VI. 


Notations: N++ and Nq denote the set of positive integers 
excluding and including zero respectively. Q++ is for the 
positive rational numbers. M„ is for all the matrices with 
dimensions n x n. C" with n S Nq stands for the set of 
n times continuously differentiable functions and C°° for the 
set of smooth functions. The gradient of a continuous function 


f &C 


Two 


1 . ^ K is v,/(a:) ^ • ■ •, ^ 

vector fields /, g : K" x K —>■ K” are twice continuously 
differentiable; their Lie bracket is defined as [f,g]{x,t) = 
f{x, t) — g{x, t). For a point x G X and a set 


S C X,x ^ S, the distance from a: to 5 is defined as 
dist(a;,iS) = min^g^ ||a: — s||. For two compact sets X,y and 
X Giy, dy for the boundary of y, the distance is defined as 
dist(A:’, 3 ^) = int ^ means the interior 

of set X. 


11. Preliminaries 

A. X-norm 

The A-norm, introduced by Arimoto et. al. in 1984 fTS) , is 
a topological measure widely used to analyze the convergence 
of ILC control law [2^ . The formal definition of A-norm is 
as follows. 

Definition 2.1: The A-norm of a function / : [0,L] — 

K" is 

ll/(•)IU = max e-^‘||/(t)||oo 
te[o.L] 

where ||/(f)||oo = maxi<i<„ \fi{t)\. 

From the definition, it is easy to see that 

||/(.)IU<||/(.)||c<e^^||/(.)|U 

for positive A, where ||/(•)||c = max^gjo^ij ||/(f)||oo- 

This shows that the A-norm is equivalent to the C-norm, 
which means the convergence with respect to A-norm is 
still valid with respect to C-norm. Its advantage is that a 
non-monotonically converged sequence on C-norm can be 
monotonically converged on A-norm for a properly chosen A. 


B. Lie bracket system 

In the classic extremum seeking literatures, for example 
0 . the behavior of the original extremum seeking system 
is analyzed by averaging. However, within this paper, an 
emerging analysis tool based on the Lie bracket approximation 
is going to be used to study the extremum seeking system. For 
an input-affine extremum seeking system 

X = bi{x)y/LCUi{ujt) + b2(x)s/oju2{ujt) 

with w S (0,oo), its Lie bracket system is 

For instance, in a traditional ES system, bi{x) = 1 , 62 ( 2 :) = 
—af{x), a > 0, f{x) S : K" —>■ K admitting a local 
minimum, its Lie bracket system is i = —aVf{z)/2, which 
clearly minimizes the cost function. One advantage of using 
Lie bracket approximation is that the approximating system 
and the original system share the same rate of convergence, 
i.e., the exponential stability of the Lie bracket system implies 
the practical exponential stability of the original one. For 
details, please refer to p4) . 

III. Problem statement 

In this paper, we study the following optimization problem; 
at each time t G [ 0 , L], we solve the 

minF(x(f),f) (1) 
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where x{t) € M” and F : M" x [0, L] —)• K, 

F{x{t),t) = fit) + [xit) - x*it)fQ[xit) - x*it)] (2) 

Q G M„ is a positive definite matrix. L > 0 is the finite time 
duration or period. For each time t, x{t) = x* (t) is the solution 
to the optimization problem Q with the corresponding optimal 
value /*(i). For all time t G [0,L], all the x*{t) form a time 
varying optimizer/minimizer (or called optimizer/minimizer 
trajectory) x* : [ 0 , 1 /] —?> M"; if x*{t) = c for any t, c 
a constant vector, then x* is named a constant optimizer. 
Similarly, the optimal trajectory f* : [0, L] —> K consists of all 
the optimal value /*(f) over the interval [0, L], x{t) in Q or 
its ensemble x is named optimization variable or simply input. 
Obviously, the study on minimization problem does not loss 
any generality; it can be easily extended to the maximization 
problem by only altering the sign of the extremum seeking 
gain. According to Ariyur and Krstic (pp. 21) |[T], the above- 
mentioned form of parameterization is able to approximate 
any vector function F(x(f),f) with a quadratic minimum at 
x*{t). The objective for extremum seeking is to steer x to the 
minimizer trajectory x* over [ 0 ,L]. 



\/lv cos{ijjt) a\/Zj saiiujt) 

Fig. 1: Block diagram of iterative learning extremum seeking 


A. Classical approach 

The input x in the conventional ES literatures, is driven to 
the neighborhood of the minimizer trajectory x* by solving 
the following ordinary differential equation (ODE). 

xit) = —aFixit),t)^/u}siniuJt) + ^/cccosiuJt) (3) 

Here a G (0,oo) is a constant gain. The black part in Eig. 
[T] presents the closed loop of this classic approach. Eor a 
fast changing minimizer trajectory, this approach may fail to 
achieve a satisfactory tracking within finite duration. 


B. Main idea 

Since the repeated operation mode of the repetitive process 
there is no need to solve the tracking problem in only 
single iteration; instead, it can be solved in many iterations 
even infinite iterations. Since the previous input is a good 
approximation of the optimizer trajectory, it is quite handy 
and natural to modify the previous input to generate a new 


control input. Borrowing the ideas from ILC, we propose to 
solve the ES tracking problem by solving the following ODE. 

Xkit) = (1—/3)i;fc_i(f)—aF(a;fc(f),t)Vwsin(a;f)+-\/wcos(a;f) 

(4) 

The subscript k G N++ indicates the iteration index; /3 G (0,1] 
is the forgetting factor, which has been adopted in many ILC 
literatures, i.e., | [T8) . To keep the notation simple, Xk will 
be used in the rest of paper instead of x^it) without any 
ambiguity and the same for other similar variables, unless 
stated otherwise. 

Physically, 0 introduces a memory storing the input (x^-i) 
of the previous batch into the extremum seeking loop as the 
red part shown in Eig. [T] The term Xk-i is the feed-forward 
component, while the second and third terms in the right hand 
side of Q are the feedback component. It is intuitively un¬ 
derstandable that the proposed method could result in a better 
performance than the conventional one (|^, since the feed¬ 
forward term somehow can facilitate the tracking. Meanwhile, 
the mechanism is always feeding a new input into the system 
by insistently using feedback information to “polish” Xk-i, 
a rough “guess” of the minimizer. Thus, it can be expected 
that the tracking performance would improve gradually as the 
rough “guess” is becoming finer. 

Remark 3.1: It is noted that when fc = 1, 0 becomes 
ii = (1 — (3)xo — aFixl,t)^/uJsiniuJt) + fuj cos(a;f), which 
is exactly the standard extremum seeking, if Xoit) and Xoit) 
are assumed to be zero, or equivalently the memory is reset 
to zero. 

Since we are only interested in the system over the finite 
interval, there is no need to discuss its asymptotic stability 
along the time direction. The problems we are more in¬ 
terested in are under what condition Xk will approach to 
a small neighborhood of x* when k tends to infinity, i.e., 
\\xk — x*\\c < F), k —i' oo and what determines D. 


IV. Main results 

Note that if 0 iterates itself to different k, the correspond¬ 
ing coefficients on cos(tLif) will be different. On the other hand, 
the Lie bracket approximation is only valid for systems with 
respect to t not to both k and t. Thus, we cannot directly 
employ it but have to tailor it accordingly. We propose to use 
the modified Lie bracket (MLB) system to approximate the 
original one. 


Zk =(1 - I3)xk-i + 4-[l,-aF](zfe,f) 

^ (5) 

=(1 - /3)xk-i - t) 

Here the only modification is the introduction of 7 ^. jk is 
a compensating parameter and only related to the forgetting 
factor as defined below. 


Ik 


l-(l-/3)'= 

/? 


(6) 


It is evident that { 7 ^} is a monotonically increasing sequence 
and 1 < 7 fe < 1//3. The first equality holds if and only if 
fc = 1, which implies that the MLB system 0 reduces to the 
traditional Lie bracket system when fc = 1. The term Xk-i is a 
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feed-forward term and only a function of time with respect to 
the current iteration k, and does not contain any sinusoid term. 
Thus, it should not be included into the Lie bracket according 
to the theory in p4) . 

Remark 4.1: Observing the MLB system Q is unlike 
the conventional Lie bracket system, because of the existence 
of the derivation of Xk-i rather than being an independent 
system of itself Inserting Q into Q and denoting 

P _ aikQ _ [1 - (1 - l3f]aQ 
^ 2 2/3 

we can rewrite 0 as 

Zk = (1 - /3)zk-i - Tk(zk - X*) + (1 - /3)(xk-i - Zk-i) (!) 

Before presenting our main results, we impose the following 
assumptions. 

• Al) The time varying optimizer a;* S (7^ : [0, L] —>■ K" 
and optimal trajectory f*GC^: [0, L] —^ M; 

• A2) The initial condition of each iteration of 0 is 
identical and equal to zero, i.e. Xk( 0 ) = x(0) = 0,VA:; so 
is the MLB system 0, i.e. Zfe(O) = a;(0) = 0 for all k\ 

• A3) Assume that Q is bounded and Q > 51, where 5 is 
a known positive real number. 

Remark 4.2: Al is assumed to ensure the existence of 
F{x,tys first-order partial derivative ^ which is re¬ 

quired by integration by parts. A2 is a common assumption in 
majority of ILC literatures to simplify derivations ig, called 
identical initial condition (i.i.c). The second part of A2 is 
assumed to let MLB system be in accordance with the original 
system. A3 means that we do not require exact knowledge 
about Q, the Hessian matrix, but requires a lower bound of 
Q for the minimum case, since a known Q means having a 
precise knowledge about the plant, which is generally impossi¬ 
ble in practice. 5 is only required by the following conceptual 
analysis, does not restrict our method’s applicability. 

Taking integration on both sides of 0, it turns to be 

Zk = {l-(3)zk-i-Tk / {zk-x*)ds+{l-(3){xk-i-Zk-i) 
Jo 

(8) 

The Zk can be interpreted as control input with Zk — x* as 
tracking error. Then, the rewriting above clearly shows that it is 
in the form of integral-type ILC online (feedback) control p4| 
with some “control input disturbance”, i.e., (1 — I3){xk-i — 

-2'Ic—l)' ® 

Note that 0 is a linear ordinary differential equation, and 
we can write down its explicit solution. 

Zk = -f (1 -/3)a;fc_i]ds 

Jo 

It follows from z(0) = 0 and integrating by parts that 

t 
0 

Jo 

Because a:fc_i(0) = 0, we have 

Zk = 0-l3)xk-i+^~^'‘* [ ( 9 ) 

Jo 


Zk =(1 - /3)e 


-Tkt 




[ e^'^'FkXk-ids 

Jo 


Now we define the tracking error of the MLB system 0 as 
Vk — Zk — X*', the error system is 

Uk = Tk (^k-i + Xk-i - Zk-i - 

where Tk is the mapping as follows. 

Tk{x){t) = [ Q^^^Tkx{s)ds (11) 

Jo 

For the sake of simple notation, we will write •tn) for short as 
Tk{x) = (l-^)a;-(l-/3)e-r''*/g e^'=®rfca;ds by abusing Tfc 
a little. Note that the mapping above is ^-dependent because of 
Lfc is fc-dependent. Now we will give the result of contraction 
mapping for fc-dependent case, which differs from pp. 655 
101 (where only fc-invariant mappings are studied). 

An operator T between two real linear spaces X and y is 
called a linear mapping or linear operator if T{Xx + py) = 
\T{x) + pT{y) for all A,/i S K and x,y G X. There is a 
norm || • || defined on X and y. Then, a linear mapping is 
bounded if there exists a constant M > 0 such that 


||r(a;)|| < M||a:|| for all x G X. 


The sets consisted of all these bounded linear mapping T is 
denoted by ^{X,y). We write B{X) for short if the domain 
and range spaces are the same. Moreover, the mapping norm 
is defined as 


|!T|| = sup 


r(:^)ii 

ii^ii 


Definition 4.1 (Uniformly convergence (pp.l09, 1^)): If 
{Tk} is a sequence of mappings in B{X,y) and 

lim \\Tk - T|| = 0 

k—^oo 

for some T G B(X,y), then we say that Tk converges 
uniformly to T. 

Note that \\Tk{x) — T(a:)|| < \\Tk — T||||a;||. Given that 
||a;|j is bounded, limfe_>oo \\Tk — T\\ =0 implies that 

linifc_>oo Tk{x) = T(x). In other words, uniform convergence 
implies strong convergence. 

Theorem 4.1 (k-dependent contraction mapping): Let S be 
a compact subset of a Banach space X. If a mapping sequence 
{Tk} satisfies 

• Cl) for every k, Tk G B(S'); 

• C2) \\Tk(x) - Tk{y)\\ < p\\x - t/||,Vx,t/ e S, for a 
universal contraction mapping ratio p G [0,1), denoting 
T as the set consisted of all such Tk', 

• C3) Tk converges uniformly to T^o as k ^ oo, for some 
Too e T, 

then Tk is called a k-dependent contraction mapping on S. 
Furthermore, 

• there exists a unique solution x* G S satisfying x* = 

Tooix*y 

• X* is independent of the initial value xi G S. 

Proof: The proof is similar to Theorem B.l in p5] . 
Arbitrarily select Xi G S and generate a sequence {xk} 
according to the formula Xk+i = Tk{xk). Every Xk G S, since 
Tk G B(S'). First, we will show {xk} is a Cauchy sequence. 
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It follows from S is compact that there is a constant D > 0 
such that Ikll < D for all X € S. Additionally, that {Tk} 
is a Cauchy sequence follows, since Tk converges uniformly. 
Then, for an arbitrary e > 0, there exists a such that 
\\Tm — Tn\\ < e/D for any m,n > k^. For k — 1 > k^, 
we have 

ll^fc+l ^fc|| -(xi;;—1 ) II 

<\\Tk{xk) - Tk{xk-i)\\ 

+ \\Tk{xk-i) — Tk-i{xk-i)\\ 

<p\\xk - Xk-i\\ + \\Tk - Tfc_i||||xfc_i|| 

<p||xfc - Xfc_i|| + e 

It follows that 


r—1 

r—1 


<X! - Xk-i\\ +p'e] 

i^O 

P W II ^ 

- \\Xk - Xk-l\\ + 


I-p 


1-p 


<T^^{p\\xk-i - Xk-2\\ + e) + T~~ 
1-p I-p 


< 


P 

I-p 


k — kf — 1 


k—k^ — 2 \ 

|a:fe,+i - XfeJI + ^ p*e 

i=0 ) 


+ 


1-/0 

pe e 

<— -1- - -1- 

1- p (1 -p)2 1- p 

2pfe-fc,^ e 

<— -1- 

- 1-p ^(l-p)2 


Since e is chosen arbitrarily, the right hand side will go to 0 
as k ^ oo. Hence, {x^} is a Cauchy sequence. From that X 
is complete, Xk ^ x* G X as k ^ oo. Furthermore, S is 
closed; it follows that x* G S. 

The second step is to prove x* = Too(x*). For any Xk+i = 
Tk{xk), it can be obtained that 


||x* - Too(x*)|| <||x* - Xfe+ill + ||Xfc+i - roo(x*)|| 
^||x X/j;-|_l|| T ||xfc+l Too(X/j;)|| 

+ llF'oo(Xfc) - Too(x*)|| 

<||x* - Xfe+ill + llTfe - Toolllixfcll 
+ p||xfc - x*|| 


It is apparent that the right hand side can be made arbi¬ 
trarily small by selecting a sufficiently large k. Thereafter, 

X* = Too(x*). 

Finally, we will show the uniqueness. Suppose that there is 
another fix point y* satisfying y* = Toa{y*)- Then, we have 


||x* - y*\\ < \\T^{x*) - T^{y*)\\ < p||x* - y*\\ 


Since p is strictly less than 1, it is a contradiction; then x* = 
y*. This completes the whole proof. ■ 

Define a Banach space X = C[0,L] and a compact set 

S = {y G X\\\yh < D 2 } (12) 


It implies that Zk is contained in = {z G A’|||z — x*||a < 
£> 2 }. The lemma below shows the conditions to fulfill C2 for 

Tk. 

Lemma 4.1: Let A1-A3 hold. For arbitrary /3 G (0,1), there 
is a Ao such that ||Tfc(x) - Tkiy)\\\ < p\\x - y\\x for any 
x,y G X, p G (0,1) for every A G (Ao,oo). Moreover, p and 
Ao are independent of k. 

Proof: For details of the proof, please refer to Appendix 

■ 

Remark 4.3: Many ILC literatures do not use such a 
forgetting factor, since its existence will comprise the zero 
tracking error (perfect tracking) iteration-wisely m- How¬ 
ever, the above lemma suggests otherwise in our case. It is 
necessary to introduce such a forgetting factor /? to ensure the 
existence of the ^-dependent contraction mapping, since that 
the perfect tracking cannot be achieved due to the existence of 
dither signal. If without /3, this undesired effect would keep 
accumulating and the overall performance would deteriorate 
rapidly. 

Remark 4.4: If the C-norm is used instead of A-norm, or 
equ ivalently A = 0, then according to the proof of Lemma 
/3 has to be greater than 2 — x/2. It suggests that A-norm 


4.1 


somehow enlarges the feasible basin of /3. 

Remark 4.5: From (11 1 , it is evident that the mapping Tk 
is a linear mapping. Thus, we can rewrite ( |T0) | as 

Uk = Tk{yk-i) +Tk{xk-i- Zk-i) +Tk 


In (13 I, it shows that the second and third terms on the right 
hand side play a role like “disturbances”; the second one 
is caused by the approximation, while the third is caused 
by the forgetting factor. The two “disturbances” are still 
different: Tk{xk-i — Zk-i) is a persistently active noise, while 
Tk[—fix*/{1 — /?)] is just a constant offset. 


A. I-type ILC 

To this end, we are ready to present our contribution on 
ILC, which studies the behavior of the disturbance-free MLB 
system (0, i.e., 

Zk = - f)zk-i -^k f (zk - X*)ds (14) 

Jo 

Or equivalently, its corresponding error system 

yk = Tk{yk-i)+Tk(^-^^x*^ (15) 

will be studied. It should be mentioned that an underlying 
assumption has been imposed, namely, x* is known, when we 
discuss on I-type ILC. Now, we dehne another mapping as 

Gk{x) ^ Tk{x)+Tk 

The objective is to show Gk is a fc-dependent contraction 
mapping. It is easy to verify that Gk satisfies C2, since that 
Gfc(x) - Gkiy) = Tk{x) - Tk{y) and Tfc(x) - Tk{y) fulfills 
C2. The following lemma gives a sufficient condition that Gk 
fulfills CL 
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Lemma 4.2: Let A1-A3 be satisfied. Given A > o,/3 e 


(0,1), /9 S (0,1), if D 2 in (12 1 satisfies 


L >2 > ma.x{Do,D*}, 
then Gfc maps S into S. D* is defined as 

/ 3 p 

m 

And Do = IIj/iIIa, Vi yielded by 


yi = -X* - Ti / yids 
Jo 


as in Lemma 4.2 there exists a unique limit solution for 


Voo - 2^00 ( Voo 


1-/3 


Too is the limit of as k tends to infinity, like 


T’oo(a;) = (1 -/3)a; - (1 -/3)e / e^ 

^0 


with 


r — 

-L OO - 


aQ 

w 


Expanding Gk{x), we have 

IIGfell < sup ||Tfc(a:)||A 
ILIU=i 


Tki- 




1-/3 


IIGfcll <p- 


Tk - 




1-/3 


< +00 


uniformly. It suffices to show Tk converges to Too uniformly. 
By definition, we have 

\\Tk-T^\\= sup ||Tfc(a;) - Too(a;)||A 
lklU=i 

From the definition of A-norm and ( [T0| , we further get that 

llTfe-Tooll < (1-^) 


X sup e 
i G [0,i] 

lkiu=i 


It is simply executing (15 i on A: = 1. 

Proof: For details of the proof, please refer to Appendix 


By the mean value theorem, it is easy to derive that 

llTfe — Too II 


Lemma 4.2 not only presents a sufficient condition for Gk 
satisfying Cl, but also states that ||j//c||a is uniformly bounded. 
Obviously, we can offer more than that, i.e., convergence and 
uniqueness of the limit solution, by invoking the fc-dependent 
contraction mapping theorem. 

Theorem 4.2: A1-A3 are satisfied. If £>2 > m.ax.{Do,D*} 


<(1-/3) sup e 
te[o,L] 
ILIIa^i 


X l|2;(0lloc,(Ce [0,£]) 


<(1-/3) sup 
te[o,L] 


/(■ 


——s)t 




ds 


in (15 I as k tends to infinity. Moreover, limfe_>.oo = Voo, 
where yao is defined as the solution to the following equation. 


(16) 


^r^xds (17) 


(18) 


Proof: It is noted from that is equivalent to 

Voo — Goo(yC3o) 

Hence, we only need to check whether Gk satisfies C1-C3 
or not. Since X is equipped with A-norm, we can define the 
mapping norm by the A-norm as 

IIGfcll = sup ||Gfc(x)||A 
lkllA=i 


On the other hand, from Lemma |4.1| it is known that 

ll'7fe(a;)||A = \\Tkix) - Tfc(0)||A < p||a:||A- It immediately 
follows that 


<(1-/3) sup e 
te[o,L] 

The term exp(—Ffcf) can approach exp(—Foot) arbitrarily 
small as fc —?► oo. Therefore, it can be concluded that 

\\Tk — 31,011 —0 as A: —^ oo 

which is the uniform convergence. Then, apply the k- 
dependent contraction mapping, we can conclude the result. 

■ 

Remark 4.6: Theorem |4.2| shows that the trajectory of the 
disturbance-free MLB system ( [T4) l will ultimately converge to 
a fixed trajectory, which is parameterized by /3 and x*. From 
0’ it is clear that 0 is a solution to the equation x = Taoix). 
From one can see that j/oo will approach 0 if /3 —> 0. But 
will j/oo be 0 if /3 = 0? From Lemma 4.1 it is known that 
the contraction mapping method will fail when /3 = 0. Thus, 
we use another way to prove that conjecture in the following 
theorem. 

Before presenting the theorem, we have to make a remark 
on this particular case ^ = 0. In the following theorem, we 
will abandon F^ but use a constant gain F (positive definite 
matrix, not necessary restricting to aQjT) instead. The reasons 
for doing so are twofold. First, suggests that F^ will be 
ill-defined, since 7^ —> 00. Second, as Remark [43l states, the 
motivation of introducing f3 is to handle the “control input 
disturbance”, and the gain F^ becomes A;-varying because of 
/3; now we are hereby dealing with “disturbance”-free control 
system. 

Theorem 4.3: Considering /3 = 0, that is Zk is generated by 
the following formula 


Zk = ife-i - r(zfc - X*). 

If A1-A3 hold and a;*(0) = 0, then 

Zk —>■ X* almost everywhere as k ^ 00. 


(19) 


Combining with Lemma 4.2 Gk G B(S') for arbitrary k. Also 


noted from Lemma 4.1 the mapping sequence {Gfc} satisfies 
Cl and C2; it remains to show that Gk converges to G^o 


Proof: From (19i, within this proof, we are equivalently 
studying 


Vk = Vk-i - Tt/fc 


( 20 ) 
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Define an index as follows. 


Jk — / ^ 

Jo 


-Xt-T- 


Vk Vkdt 


( 21 ) 


where A > 0. Rewriting (20i as yk-i = yfc + Tt/fc, we insert 


it into (211 and compare the difference of (211 between k and 
k-1. 

fL 

dk Jk—1 — I 


^ — Xt 


[ylvk - ivk + Tyk)'^{yk + ^yk)] dt 


^-Xt^T-rTj 


yf.r Tykdt-2 / e yk^ykdt 
Jo Jo 

As for the second term in the right hand side, we integrate by 
parts and derive that 


2 / e ^*ylrykdt=e ^^y^Tyk 
Jo 

Thus, we have 

Jk Jk—1 — 




+ ^ e yu^Vkdt 
t=o Jo 


e-^*yl (r^r + Ar) y^dt 


— e 


0 

-Xt^T 


yi iL)Tyk{L) 

L 


<- Pn 




Vk Vkdt 


Let pinin be the smallest eigenvalue of the matrix F^F + AF. 
It is apparent that pmin > 0 since F is positive definite and 
A > 0. Thus, {Jk} is a non-increasing real number sequence, 
and Jfc is bounded below by 0, Jk converges. It follows that 


lim 

k—>-(X> 


Vk Vkdt = 0 


It is the L2-norm of yk- Thus, we can claim that yk converges 
to 0 almost everywhere. ■ 

Remark 4.7: Theorem 4.3 gives a weaker result than 


Theorem 4.2 since it can converge to 0 except on a set 


whose measure is 0 and the uniqueness is lost. The result is 
quite understandable from a perspective of Laplace transform. 
Taking Laplace transform on both sides of ( |20| ), we have 

Yk{s) = 

The modulus of the gain is less than 1, i.e., |s/(s + F)| < 1. 
But it tends to 1 as s —oo. It suggests that this algorithm 
have a weaker decaying effect on high-frequency signal. It 
coincides with the result. 


B. Iterative learning extremum seeking (ILES) 

We follow the similar idea to study the dynamics of ILES. 
ILES is an ILC control policy with “control input disturbance”. 
Due to the “disturbance”, the unique limit solution cannot be 
achieved. Therefore, ILES converging to a set will be shown 
instead, if the “disturbance” is bounded. 

According to ( [T3] l, we define a new operator Hk as 

Hk{x) ^ Tk{x) + Tfe[-/3x7(l - /3)] + Tk{xk-i - Zk-i) 

Since Tk is a bounded operator, so is Hk provided that Xk-i — 
Zk-i is bounded. It is obvious that Hk {x)-Hk{y) = Tk{x)- 


Tk{y)\ it is easy to verify that Hk satisfies C2. Following the 
procedures we did in Section IV.A, we are going to show that 
how to properly design D 2 to ensure Hk maps S into S. 

The following lemma lays the foundation of inductive 
arguments towards that conclusion. It basically means that for 
any k we can always ensure that Xk is in a invariant set given 
that its MLB system Zk is in the interior of the invariant set, 
if xi,X 2 ,. ■., Xk-\ are all in that set. This can be achieved by 
selecting a frequency uj larger than a threshold loq, which is 
independent of k and t. 

Lemma 4.3: Let A1-A3 be satisfied and suppose that 
xi,X 2 ,. ■., Xko -1 are uniformly in t contained within a com¬ 
pact set S G K" and 0 € int S. If Zk,, is contained in 
/C C int 5, 0 S /C, then there exists a ujq G (0, -|-c») such that 
for every utg G (wojOo), Xkg(t) is uniformly in t contained 
within S as well for any /3 G (0,1). Moreover, dist(/C,5) can 
be made arbitrarily small by selecting a sufficiently large iv. 

Proof: This proof uses the similar arguments as Theorem 
1 in | |24l , but tailors them for the iterative case. For details of 
the proof, please refer to the Appendix [C| ■ 

The lemma above also indicates that the “disturbance” 
\\xk — Zk\\\ is uniformly bounded; it can be arbitrarily small 
by selecting a sufficiently large uj. 

Theorem 4.4: Let A1-A3 hold. Given A and p G (0,1), there 
exists a Wo G (0,-foo) such that for every w G (wo,oo), Hk 
maps S' into S, if £>2 in ( [T2| 

L>2 > inax{L)Q, D*} 


where Dg is defined in Lemma [LT] and 


D* 




Di is the uniform bound of |ja:fe — Zk\\\ for an arbitrary k. 
Proof: We are going to show the theorem in an inductive 

way. 

First, at fc = 1, both the extremum seeking system (ffl and 
the MLB system (j^ are in the standard form. From (j^, the 
explicit solution of the MLB system (j^ is 

e-ri(‘-")Fix*ds 

Because all the eigenvalues of —F1 lie in the open left complex 
plane and x* is contained in a compact set, zi is well defined 
for initial condition zi (0) = 0 in a compact set over the time 
interval [0,L]. According to the definition of D 2 , it is known 
that IIj/iIIa < D 2 - Thus, by Lie bracket theorem p4) , it is 
known that the distance between xi and zi can be arbitrarily 
small provided that the frequency w is sufficiently large. 
Hence, there exists a wi such that for every w G (001,00), 
we have \\xi — zi\\x < Di. 

Now, we assume that ||t/i||A < £>2 \\xi — Zi\\x < Di 
for any i = l,2,...,fc—1 over the entire time interval. Then, 
we will show that these two relations are still valid for i = k. 
From Lemma |4.3| it is easy to conclude that there exists a 002 
such that for every oj G (a; 2 ,+oo), \\xk — Zk\\\ < Di. We 
only need to show HyfeUx < D 2 - 
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By the linearity of we have 


A = 


< 


Hk{yk-i) - Hk{0) 
( Xk—\ ^k—1 


1-/3 


( ^k—l ^k — 1 


1-/3 

+ \\Hk{yk-i)-Hkm\x 


X* - Tfe(O) 


<p\\yk-i\\\ + p 


^k—1 ^k—1 


1-/3 


<pD2 + pi Di + 




1 - 


a;*|U <D2 


Therefore, by selecting wq = max{a;i, 072 }, we have ||//fc||A < 
D 2 for all k. This completes the proof. ■ 

Theorem |4.4| suggests that the uniform bound of tracking 
error of the MLB system 0 - D 2 , can be made small 
by reducing the approximation error (Di), i.e., employing a 
sufficiently large w. 

Since Xk — Zk is consistently varying, it is impossible to 
show that Hk satisfies C3. Therefore, we cannot conclude the 
unique limit solution; however, we can show that yk converges 
to a A-norm ball. 

Theorem 4.5: A1-A3 are satisfied. Given X, f3 G (0,1) and 
p G (0,1), there exists a ujq such that for every w G (wq, 00 ), 
yk in (0 will converge to a set y as k tends to infinity. 
Furthermore, 


y = {y&S\\\y-y^\\x<Dy} (22) 

Here y^o is defined in ( [T^ , Dy = pDi/{l — p)\ Di is the 
uniform bound of \\xk — Zk\\\ for an arbitrary k. 

Proof: The proof is in Appendix ■ 

Remark 4.8: Theorem |4.5| suggests that we cannot achieve 
“perfect tracking” or a fixed limit trajectory unlike many ILC 
control laws, due to the existence of dither signals (sinusoid 
signals). However, according to Lemma [43j Di can be made 
arbitrarily small for a sufficiently large uj. Thus, so is Dy, 
since Dy is proportional to Dp, that means we can make the 
ultimate trajectory be as close to a fixed limit trajectory as one 
wishes by selecting a sufficiently large frequency. In the mean 
time, we can also let the fixed trajectory be as close to zero 
as possible by having a small enough forgetting factor /3. 


V. Illustrative example 

A. I-type ILC 

Consider the following tracking reference for the ILC con¬ 
trol law in ( fl4| ). 

Fig. 1^ shows the trajectory evolution versus iteration for the 
I-type ILC with a forgetting factor /3 = 0.5. It clearly verifies 
that the trajectory (zk) will converge to a fixed trajectory, 
but the trajectory has a gap with the tracking reference. Fig. 

demonstrates the evolution for the I-type ILC without the 
forgetting factor /3. It is shown that the trajectory will converge 
to the reference ultimately, thus “perfect tracking” achieved. 


Meanwhile, it proves the result of Theorem 4.3 




B. ILLS 

We study the following static map. 


(—t) 

V20 J 


y = x^ + 2 sin [ — 

It is evident that the minimize!' trajectory is x* = — sin (^f)- 
As for the iterative learning extremum seeking control law, 
a is selected as 0.1 and the forgetting factor /3 = 0.3. Figs. 
|4apd] shows the evolutions of the original system and the 
MLB system 0 in the 1®^*, 2"“*, 50* iterations under 

the frequency oj = 7. They indicate that the tracking perfor¬ 
mances of both the systems improve gradually, although the 
fluctuations are getting larger but finally bounded. Figs. 5a- 
[5d] shows the similar evolutions under the frequency ui = 15. 
Comparing Figs. |4apd| and Figs. [5apd] it implies that both 
the tracking error (the fluctuation of the MLB system Q) and 
the approximation error (the gap between the original system 
and the MLB system 0) are smaller under a larger frequency 


VI. Conclusion & Outlook 

This paper has proposed an iterative learning extremum 
seeking approach to solve the time varying optimizer tracking 
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0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 

Time (s) Time (s) Time (s) Time (s) 

(a) 1^* iteration with oj = 7 (b) 2"*^ iteration with u — 7 (c) 3^^^ iteration with oj — 7 (d) iteration with lj — 7 

Fig. 4; The evolution of the original system and MLB system under frequency cu = 7. 



3 
2 
1 
0 
-1 
-2 
-3 

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 

Time (s) Time (s) Time (s) Tiine (s) 

(a) iteration with a; — 15 (b) 2"^^ iteration with a; — 15 (c) 3^*^ iteration with a; — 15 (d) 50'^ iteration with oj — 15 

Fig. 5: The evolution of the original system and MLB system under frequency ui = 15. 



problem. A modified Lie bracket system has been introduced 
to study the behavior of the ILES system. It has shown that the 
MLB system is an online integral-type ILC control law with 
bounded “control input disturbance”. The convergence of the 
corresponding ILC control law has been analyzed. Based on 
that, the convergence of the proposed ILES to a set has been 
shown. The size of the set is reducible by tuning the frequency 
of the dither signal. The distance between set’s center and 
original is also tunable by some appropriate forgetting factors 
/3. In the future, it is quite interesting to investigate how to 
extend the method to cover a general function F{x,t). It is 
worthy to study the dynamic mapping situation as well. 


The first term on the right hand side is exactly (1 —/3)||a; —?/|| a 
according to the definition of A-norm. Eor the second term, we 
do the following operation. 

||rfe(a;) - Tkiy)\\x < (1 - ^)||ai - t/IU + niax e"^*(l - /3) 

te[o,L] 

e-r.t /■ 

•^0 CXD 

According to mean value theorem, it can be obtained that 


Appendix A 
Proof of Lemma ITTI 


Proof: Erom the definitions of Tk and A-norm, we have 


\\n{x)-n{y)\\x 

= max e“^‘(l — B) 
ie[o.L] 

(x - y) - f e^'=®rfc(x - j/)ds 

Jo 


It follows from the triangle inequality of norm and max{a + 
b} < max{a} + max{6} that 


; f ^^Tk{x — y)ds 

Jo 

.-Tkt f Qi^k+>^P^rkds - y{0)) 

Jo 


Ke (o,f)] 

<e-^«||x( 0 -t/( 0 llc 
[?e (o,f)] 


< max {e - 2/||oo} 

sG[0.t] 


-Tkt / Q(^k+xi)^Ykds 


-r,i / QtPi^+^P^Ykds 


m{x)-n{y)\\x 


< max e ^‘(1 -/3)||x - j/||oo 

tG[0,L] 

-I- max e“'^*(l — /3) 

tG[0.L] 


1 — Tfci 


[ e'^^^rk{x-y)ds 
Jo 


The first inequality is followed from the definition of induced 
matrix norm. It is also noted that 


max {e ^"||x - y||oo} < ||a; - 2 /|j a 

sG[0,t] 
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Hence, 

\\n{x)-Tk{y)\W 


<(1 - I5)\\x - y\\x ( 1 + max e 

tG[0,L] 


-At 


From ( [T5| ) and Lemma [TT] we have 

llj/fclU =||Gfc(j/fc_i)||A 
< 


.-rkt / Q{^k+xi)sY fis 


<(1 - (3)\\x - y\\x 1 + max 
te[o.L] 


- P)\\x - y\\x 

(Ffe + AJ)-i 


1 + max 
tG[0,Z,] 


Jo 

|'^_e-(r.+A/)t^ Tfc 


Tkivk-i) - Tfc(O) + Tk - Tk{0) 

0)I|a 

a:*VTfe(0) 


<||Tfc(2/fc_i)-Tfc(0)|U 


n - 


1-/3 


<p\\yk-i\\x + p 

P 


P 


<pD2 + p 


1- 


1-/3 

:X* 


From the statement, we have Do > 


II^IU-Thus, 


For a matrix A e M„, < ||^||oo < x/^WAy, 

which is obtained from the norm equivalence theorem. Then, 
it follows that 


(^I_e-irk+M)t^Yk{rk + Xir 

II 


Iz/felU < pD2 + p 


p 


1 -, 


< pD2 + (1 — p)D2 = D 2 


/n 
<x/n 


rfe(rfc + A/)- 1,2 
Thereafter, we have 

\\Tkix)-Tkiy)\\x < (l-^)||a:-2/|U(l+Vn||rfc(rfe+AJ)-i2) 

Note that ||rfc(rfc + A /)“^||2 = ||/-(rfc + A/)“^|| 2 , it is a non 
increasing function of Ffe. From the assumption, it is known 
that Ffc > aS/2I. Denoting p = (1 — /3)(1 + x/naS/{a6 + 


Hence, Gk{yk-i) G S, and this completes the proof 


Appendix C 
Proof of Lemma I43] 

Proof: The proof is adapted from Theorem 1 in | [24) . The 
major differences are: first, p4) dealing with more general 
form - input affine system, we only focus on sinusoid input 
signal; second, p4| suitable for infinite time horizon and 
single iteration, we are handling the case of finite time horizon 
and multiple iterations. 



Since all the eigenvalues associated with Fi locates in the 
right complex plane and x* is continuous over the interval 
[0,L], zi is well defined and bounded over [0, L]. Hence, 
||t/i||A is bounded. Furthermore, it is evident that ||yi||A < D 2 , 
equivalently, yi G S. 

Assume that yk-i G S or ||yfc_i||A < D 2 holds for the 
k — 1-th iteration. It remains to show that y^ G S. 


The basic idea to show the lemma is illustrated in Fig. 
Xk<ko ^re within S and Zkg is within JC. We construct a tube 
with radius E along the trajectory of Zkg- If E < dist(/C,S') 
and Xkg is within the tube, it can conclude that Xkg G S. 
Moreover, supposing the Xkg leaves the tube at arbitrary time 
Ie, if we can show is not the time that Xkg leaves the tube, 
then we can claim that Xkg will never leave the tube. 
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Evaluating 0 , 0 at k^. Subtracting 0 from 0 and 
integrating on both sides, we have 

Xko - ZkQ= - a / F{xko ^ sin{ujs)ds 

Jo 


Note that these bounds are uniformly bounded in t. Thus, we 
have 


a7fco 


/w cos{ujs)ds 
[ VzF{zkQ,s)ds 

Jo 


Let Ra be the first term on the right hand side of 
integration on Ra by parts, we obtain that 


(23) 


; taking 


-Rllloo — 


p2||oo = 


p3||oo = 


ZZ [F{xk, s) cos{uJs)]\l 

UJ 


< 


2 aei 


a /■* dF{xk,s) 

cos(ojs) -- as 

tujn ^ ^ dt 


fiO 


aea 
- a;3/2 


2 Jfl 




ko-i 


sin{2u;s)VxF(xk, s) 


i=l 


F{xi, s)ds 


< 


2 ko 

a ei €2 


2uj 




ko-i 


2 = 1 


d I Oi. i 

i?a [^’(a:fco)'S)cos(u;s)]|o- -j= / cos{oJs)F{xko, s)ds 

Jo 


< 


U! 

= ZZ [F(a:fco,s)cos(ws'’"* 

WUJ 


a eie 2 
2/3a; 


0 


a / 

ZZ Jo 


dF{xko,s) 

cos[Lus) - — -as 


a f 

- -J= / COs{oJs)\7a;F{Xko,s)Xkods 

Jo 


ko 


2=1 


Inserting (24i into Ra, it becomes 


Ra =Ri F R 2 F Rs - a^ko / { 00 s)VxF{xko, s)ds 

Jo 


—Ri F R 2 “i” R 3 F R 4 — 




V xF{xkQ : s^ds 


Here Rri 


-EE(i 


s)||oo < ei, ||V 3 ;i^(x, s)||oo < £ 2 , 


dF{x, s) 


dt 


< £3 


-R 4 II 00 = 


aiko 


cos{2(jjs)\7xF{xko,s)ds 


< 


aiko(-2 

4a; 


< 


cr£2 


Il-Rslloo — 


Afduj 

sin(a;s) 


< 


Denote the and 2"3 terms in the right hand side of the above 
equation as Ri and R 2 respectively. Iterating 0 to the first 
iteration, it follows that 


Xka = - ’^0-/3)^° ^aF{xi,t)^/u} sm{ujt)+"/ko\Z} cos{u}t) 

(24) 


So there must exist a positive number M such that Fi < 
^ and M is independent of k. 

Since F{x,t) are twice continuous on 5 x [0,f£;], there 
exist a positive number K such that \\'VxF{xkg,s) — 
V^F(zfco,s)||oo < K\\xko - -^feolloo according to Lemma 3.2 
(pp. 90, p^). Thereafter, for t G [0,^^] 

„ „ M aK „ , 

W^ko ~ - 2 ^fcol|oo E F J W^ko ~ ^koWoods 

From Gronwall-Bellman inequality (pp. 651, p5|), we have 


I ^ko ^ko II cxD E 


M 


[0,tE] 


(26) 


and 


/3)^« * fo sm(2uJs)VxF(xko, s)F(xi,s)ds 
Ri = cos(2u>s)VxF(xko, s)ds. The second 

equality stems from the identity cos( 2 a;s) = 2 cos^(a;s) — 1 . 
Hence, Xkg — Zkg becomes 

5 „t 

Xkg-Zkg / ^xF{xkg,s)-\I^F(zkg,s)\ds 

i=i ^ Z 

(25) 

where i ?5 = sin(a;f)/-ya;- It should be noted that (25 i holds 


For any ojq G (M^e^ /E^,oo), where uiq is independent 

of k and t, Wxkgits) — Zkg{tE)\\oo < E, which contradicts. 
Thus, Xkg will remain in the tube of Zkg- Since Zkg stays in 
1C, Xkg will stay in S. ( [2^ suggests that \\xk — Zk\\x can be 
small enough if uj is large enough. This completes the proof. 


Appendix D 
Proof of Theorem I4.5I 

Proof: From Lemma |4.3| and Theorem |4.4| we know that 
there exists a oiq for every oj G (a;o, +oo) such that we can 
ensure that \\xk — Zk\\\ < Di and yk G S for any k. 


universally for any t. Now we are going to show Xkg will 
remain in S over [0, L] by contradiction. Let E — dist(/C, S). 
Assume that there is a time instant Ie G (0,L) such that 
\\xkg{t) - Zkg{t)\\oo < E for any t G [0,tE) and ||xfco(f£;) - 
Zkg{tE)\\oo = E. It means that tE is the first time when Xkg 
leaves a tube with Zkg as center and radius E. Since 5 is a 
compact set, for any x G S, there always exist £i, £ 2 , £3 such 
that 


By similar arguments in the proof of Theorem 4.2 from the 
definition of limit, for an arbitrarily chosen e > 0 , selecting 
an e' > 0 satisfying 2 e'/(l — p) < e, 

2e'D2 


I- p 


< £ 


there exists a k^ such that HGfc —GooH < can be guaranteed 
as long as fc > fcg. 

From ( |22) l, 3^ is a A-norm ball with y^o as its center and 
Dy as its radius. Hence, we have 

dist(yfc, 3 ^) = max{|jj/fc - ?/oolU “ Dy, 0 } 
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If dist(yfc,3^) = 0, it means that € 3^. 


For arbitrary k > we have the following from (10 1 ,( 16 1 . 


Wvk - 2/oo|U 

l) ^oo(yoo) “t“ k—1 i)||a 

^\\Gk{yk-i) — Gk{yoo)\\\ + \\Gk{yao) — Goo(j/oo)IU 
+ \\Tkixk-i — Zk-i) — Tfe(0)||A 
<p\\yk-i - 2/00IIA + 2e'D2 + pDi 
Iterating the above equation to k^, we have 


/c—fcg —1 

l2/fc - 2 /ooIIa <P*'"'''ll2/fe, - 2 /oo||a + ^ 2p^e'D2 

i=0 

2e'D2 


<P 'II 2 //C, -2/oo||a + 


_ pDi 

I - p^ I - p 


It follows from Theorem 4.4 that \\ykt — 2/oo||a is bounded by 
2D2. Therefore, it can be guaranteed that \\yk — 2/oo||a < Dy+e 
for any k satisfying 


k> ke + logp ( e - 


1 

2 D 2 


2t'D2 
I- p 


e is chosen arbitrarily; thus, we can conclude that 

pDi 


It follows that 


lim \\yk - 2/oo||a < , 
k—^OQ i — p 


lim dist(yfc,3^) = 0 

k—¥oo 


That means yk will finally be within y. This completes the 
proof ■ 
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